Image processing method and system for iris recognition

ABSTRACT

A method of iris recognition comprises detecting a body region larger than and comprising at least one iris in an image and performing a first eye modelling on the detected body region. If successful, the result of first iris segmentation based on the first eye model is chosen. Otherwise, a first iris identification is performed on the detected body region. If successful, the result of second iris segmentation based on a second eye modelling is chosen. Otherwise, second iris identification is performed on the image, third eye modelling is performed on the result of the second iris identification, and third iris segmentation is performed on the result of the third eye modelling. If successful, the result of third iris segmentation based on a third eye modelling is chosen. An iris code is extracted from any selected iris segment of the image.

FIELD

The present invention relates to an image processing method and systemfor iris recognition.

BACKGROUND

The iris surrounds the dark, inner pupil region of an eye and extendsconcentrically to the white sclera of the eye.

A. K. Jain, A. Ross, and S. Prabhakar, “An introduction to biometricrecognition,” IEEE Trans. Circuits Syst. Video Technol., vol. 14, 2004discloses that the iris of the eye is a near-ideal biometric.

For the purposes of iris based recognition, an image of a subjectcomprising an iris region is acquired with an imaging system, typicallyusing infra-red (IR) illumination to bring out the main features of anunderlying iris pattern.

Then, eye and/or iris detection is applied over the whole acquired imageto identify a region of the image containing the iris pattern. Irissegmentation is performed on the detected region of the image in orderto define an iris segment, and then feature extraction is performed onthe iris segment. The extracted features can be used to generate an iriscode for the subject of the acquired image and this can be used inconjunction with stored iris code(s) to identify, recognise orauthenticate the subject of the image.

If there are certain requirements for the input image regarding the irissize and location, as in the case of BS ISO/IEC 19794-6:2005 compliantimages, this can speed up the detection process.

Without such requirements, there can be a large variation in iris sizeand location within acquired images. For example, in the context ofhandheld devices, for example, smartphones, the range of distancesbetween the eye and the device can vary from less than 15 cm out to 40cm or more, depending on how close a user holds the device or the lengthof the user's arm—this can affect the size, location and quality of theiris region appearing in an acquired image.

Thus, the detection process can be slow and possibly lead to falsecandidates, since the initial processing step has to detect where theiris is located within the whole acquired image and determine the sizeof the iris.

As indicated, in an attempt to speed up processing, some systems performrelatively large-scale and so faster, eye detection before performingrefined iris detection on the result of the eye detection. However, thiscan result in non-segmented or wrongly segmented iris images, especiallyin cases where the image is so closely acquired that only a portion ofthe eye is included therein. In these cases, since the full eye is notwithin the image, the eye detection can fail to correctly locate theeye, thus providing a poor quality or wrong result for the followingimage processing.

SUMMARY

According to a first aspect of the present invention there is provided amethod of iris recognition according to claim 1. There are also providedan image processing system and a computer program product according toclaims 13 and 14.

The method combines eye detection, eye modelling, iris detection andiris segmentation to provide iris recognition which is fast andreliable, even working on input images presenting a high range ofvariation in iris size and location.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will now be described, by way of example,with reference to the accompanying drawings, in which:

FIG. 1 illustrates a method according to an embodiment of the presentinvention;

FIG. 2 illustrates an image processing sequence of the methodillustrated in FIG. 1, comprising: eye detection within an acquiredimage, eye modelling, iris segmentation and iris detection within adetected eye region; and

FIG. 3 illustrates another image processing sequence of the methodillustrated in FIG. 1, comprising: an acquired image, irisidentification within the acquired image, eye modelling and irissegmentation.

DETAILED DESCRIPTION

Referring now to FIG. 1 there is shown a method 10 for performing irisrecognition according to an exemplary embodiment of the presentinvention.

The method 10 comprises an initial step 11 of acquiring an image,followed by attempting to detect a body region larger than andcomprising at least one iris within the acquired image, step 12.

In the FIGS. 2 and 3, the exemplary acquired images 150 extend toinclude an eye region and only an iris region respectively.

Note, however, that this method makes few assumptions about the subjectbeing imaged and as will be appreciated from the discussion below, whilethe image acquired might have a field of view large enough to acquire animage of an iris and/or an eye or face or body region surrounding theiris, the image may also be so closely captured as to only extend acrossan iris region.

The only limitation on the usable field of view is that any iris imagedwithin the field of view needs to be large enough (in terms of pixelarea) to facilitate extraction of an iris code suitable for identifying,recognizing or authenticating a subject. As discussed in PCT ApplicationNos. WO2015/150292 (Reference: FN-395-PCT), the disclosure of which isincorporated herein by reference such it is generally considered that animaged iris would need to extend across more than 120 horizontal pixelsin order to be used for biometric authentication, although it ispossible to use imaged irises extending across as few as 50-100 pixelsfor authentication.

The detection at step 12 can be any type of detection suitable fordetecting image features larger than and comprising one or more irisregions.

For example, the detection at step 12 can be an eye detection resultingin one or more detected eye regions within a face, as in the exemplaryeye region 100 illustrated in FIG. 2.

There are a number of approaches to detecting an eye region within animage and these can be based on traditional type classifiers such asHaar classifiers, active appearance models (AAM) or more recentlyclassifiers such as random tree classifiers (RTC) or neural network typeclassifiers.

The detection at step 12 can also be for example a face detection, or itcan comprise face detection followed by eye detection.

Face detection in real-time has become a standard feature of mostdigital imaging devices and there are many techniques for identifyingsuch regions within an acquired image, for example, as disclosed inWO2008/018887 (Reference: FN-143), the disclosure of which isincorporated herein by reference. Again, recently, it has become morecommon to implement face detection or indeed body detection based onneural network type classifiers as disclosed in PCT Application Nos.PCT/EP2016/063446 (Reference: FN-471-PCT) and PCT/EP2016/081776(Reference: FN-481-PCT), the disclosures of which are incorporatedherein by reference.

In any case, in response to a detection of the body region at step 12,the method proceeds by performing a first eye modelling on the detectedbody region, step 13.

Eye modelling determines a number of cardinal points 103, FIG. 2, aroundthe eye within a region of an image. A number of possible techniques canbe used to determine the location of a set of cardinal points in animage or a region of an image. For example, Constrained Local Models(CLM) such as discussed in “A comparison of shape constrained facialfeature detectors”, D. Cristinacce and T. F. Cootes, Proc. Int. Conf onFace and Gesture Recognition, 2004, pp. 375-380, involves any method inwhich a set of local models are used to generate response images, with ashape model then being used to search for the best combined response.Other techniques include Supervised Descent Method (SDM) as disclosed in“Supervised Descent Method and Its Applications to Face Alignment”,Xiong et al, IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2013.

A first iris segmentation is then performed, step 14, based on thecardinal points resulting from first eye modelling step 13. Irissegmentation results in a segmented iris which is ideally preciselylocated within the region 100 detected at step 12. In the embodiment,the segmented iris is defined by an inner perimeter 104-1 and an outerperimeter 104-4.

Details of how the iris segment located between the inner 104-1 andouter perimeter 104-2 is used to extract an iris code and how such codesare employed, for example, in a biometric authentication unit (BAU) torecognize and/or authenticate a user can be found in PCT ApplicationNos. WO2015/150292 (Reference: FN-395-PCT), W02011/124512 (Reference:FN-458-EP) and WO2008/050107 (Reference: FN-457-EP), the disclosures ofwhich are incorporated herein by reference.

In step 15, the method continues by determining whether a confidencelevel associated with the segmented iris is above a predetermined firstthreshold. This confidence level can be determined either based on theiris segment itself or it can be based on iris code extracted from theiris segment.

It is known for iris code extractors, such as the BAU described inWO2015/150292 (Reference: FN-395-PCT) to associate a confidence levelwith an extracted iris code indicating the quality of iris code. Somefactors which can affect this confidence level include whether or notthe subject may have been blinking or partially blinking when the imageis acquired and so, if only a limited portion of the iris was visible orwell illuminated when an image was acquired, a lower confidence levelfor the extracted iris code is more likely. Other factors which caninfluence the quality and accuracy of the iris code are the precisionwith which the perimeters 104-1 and 104-2 have been determined. Thus, ifin order to speed up the detection step 12, only a limited number ofclassifier scales (sizes) have been used and applied at only a limitednumber of image locations, the detected eye region 100 (or face region)might not correspond closely with the actual region within the image andthis may not enable eye modelling to precisely locate the cardinalpoints 103, and so affect the accuracy of the perimeters 104-1 and104-2. Also, if a more relaxed classifier is used, again to speed up thedetection step 12, it is possible that a false location may be chosenfor the region 100. Also, if an acquired image is not well focused, theperimeters 104-1, 104-2 might not correspond closely with the boundariesof the iris segment—or indeed the iris pattern within the iris segmentmight not be easily resolved.

Alternatively, characteristics of the iris segment itself can be used toprovide the confidence level directly and even without or beforeattempting to extract an iris code. So for example, the size of the irissegment could be used as a factor in determining confidence level—ifthis is large (in pixel area or diameter), then the chances ofextracting a valid iris code are larger than for a smaller iris segment.If image contrast within the iris segment is high, then again thechances of extracting a valid iris code are larger than for a lowercontrast iris segment.

In any case, in the present embodiment, the confidence level provides anindication of quality of iris code extracted or extractable from withinthe segmented iris.

If the confidence level exceeds a threshold, at step 16, either the iriscode extracted from the segmented iris resulting from the first irissegmentation is selected or the segmented iris can be used to extractedthe iris code according to whether a confidence level based on the irissegment or iris code has been employed in step 15, and the method stops.

As will be seen from the description above, confidence level can bebased on a number of aspects of the iris segment and/or iris code and sothe test at step 15 need not be a simple scalar comparison and indeed aplurality of criteria can be employed to determine whether an iris codefor the iris segment identified at step 14 is to be used for theacquired image.

In practice, this aspect of the method 10 potentially speeds up irisrecognition by first attempting eye modelling (step 13), directly on theresult of eye, face or even body detection (step 12), i.e. featureswithin the acquired image 150 that are larger and faster to detect thanirises.

If first iris segmentation based on is detection is determined to havefailed, because the confidence score is below the first threshold, thismay have resulted from a poor eye modelling at step 13 and/or a lowquality detection at step 12.

In this case, the method 10 proceeds at step 17, by performing a firstiris identification on the region 100 detected at step 12.

Because this iris identification step 17 operates on an eye or face orbody region 100 of a known size and location, the ranges of scales andpotential locations for the iris region 102 can be more limited than ifiris detection were performed on an entire image without any knowledgeof eye location (possibly inferred from a detected face or body). Thisenables a finer set of scales and locations to be employed, so moreaccurately locating the iris region 102, but still this irisidentification step can be performed much faster than doing so directlyon an entire acquired image.

Now, a second eye modelling based on the iris region 102 detected by thefirst iris identification can be performed, step 18. This modelling canoperate in the same manner as the eye modelling of step 13, except that,as it is based on a likely more accurately located iris region 102, thecardinal points 103 returned by this eye modelling are more likely to bemore accurate than those returned by the eye modelling of step 13.

At step 19, similar to step 14, a second iris segmentation based on thecardinal points 103 resulting from the second eye modelling isperformed.

Again a confidence level associated with the iris segment produced bystep 19 is used to determine whether an iris code based on this irissegment can be used, step 20.

If the confidence level exceeds a threshold, at step 21, either the iriscode extracted from the segmented iris resulting from the second irissegmentation is selected or the segmented iris can be used to extractedthe iris code according to whether a confidence level based on the irissegment or iris code has been employed in step 20, and the method stops.

In the event that the second iris segmentation is determined to havefailed; or in the event that the detection at step 12 failed withoutproviding results, the presumption is that something was wrong with thedetection performed at step 12, for example, either that no region 100was detected or a false region 100 was detected.

If a subject image has been acquired so far away from a camera that auseful eye region cannot be imaged, then there will be little point inscanning an entire image with a small scale iris classifier, as it willbe appreciated that this might need to be applied at a large number oflocations by comparison to the number of locations used to detect theregion 100 or to detect an iris region within the region 100.

On the other hand, as illustrated in FIG. 3, if the subject image hasbeen acquired so closely that an eye (or face or body) region could notbe detected at step 12, the method continues by attempting to detect aniris directly within the image.

Thus, in the event that the second iris segmentation is determined to befailed at step 20 or in the event that the detection at step 12 failed,the method 10 proceeds by performing a second iris identification on thewhole image, step 22, rather than only on a detected image portion asfor the first iris identification performed at step 17. In this case,iris identification is based on the assumption that the iris to bedetected is large (recall that the iris scales for the identification ofan iris region with the region 100 in step 17 were smaller than thedetected region 100). Thus only a limited number of window locations ata limited number of scales need to be checked within the image and soeven if second iris identification step 22 is required, this does notincrease the processing requirement for an image greatly.

Assuming an iris region 102 is identified in step 22, a third eyemodelling based on the iris region 102 detected by the second irisidentification is performed. This third eye modelling can be the same asperformed in steps 13 and 18. Third iris segmentation based on thecardinal points 103 resulting from the third eye modelling can then beperformed, step 24, again similarly to the segmentation of steps 14 and19.

The method 10 further proceeds by determining whether the third irissegmentation has successfully executed.

Again a confidence level associated with the iris segment produced bystep 24 is used to determine whether an iris code based on this irissegment can be used, step 25.

If the confidence level exceeds a threshold, at step 26, either the iriscode extracted from the segmented iris resulting from the third irissegmentation is selected or the segmented iris can be used to extractedthe iris code according to whether a confidence level based on the irissegment or iris code has been employed in step 25, and the method stops.

In the event that the third iris segmentation is determined to havefailed, the method can either assume that there must have been an iriswithin the image; or that the image does not actually contain a usableiris. For the former, at step 29, the iris segment from whichever of thefirst, second or third iris segmentation produced the highest confidencelevel can be chosen as the segment from which an iris code for the imagewill be extracted.

This can be for example the case where the method 10 is performed, fortesting, training or recognition purposes, on a set of images which areknown to contain each at least one detectable iris, i.e. ISO images orimages acquired closed to the subject.

In absence of the above assumption on the image, the method 10 caninstead proceed by returning an indication that no iris was detectablewithin the acquired image, step 28.

This could occur either because no iris is effectively present into theimage 150 or it is too small to be detected (as in the case that theimage is acquired very far from the subject).

The above disclosed method 10 can be carried out through the execution,by a generic software processing unit, of software instructions storedon a computer readable medium of a computer program product.

For example, the instructions can be stored into and executed by imageprocessing means of an image processing system, wherein the images to beprocessed can be acquired through an image sensor of the system.

Alternatively, the method can be executed in a dedicated hardware moduleof an image processing device as part of or in conjunction with abiometric authentication device.

1-14. (canceled)
 15. A method of iris recognition, comprising:detecting, with an image sensor, a body region comprising at least oneiris of an eye in an image; performing a first iris recognition of theiris; performing a second iris recognition, the second iris recognitioncomprising a first iris identification on the image at a first detectionscale; and performing a third iris recognition, the third irisrecognition comprising a second iris identification on the image at asecond detection scale, wherein the first detection scale is relativelylarger than the second detection scale.
 16. The method of claim 15,wherein: the first iris recognition comprises: identifying a firstplurality of cardinal points around the eye based on a first eyemodeling of the detected body region; performing a first irissegmentation to identify a first segment of the iris within the bodyregion based on the first plurality of cardinal points; and determiningthat a first confidence score associated with the first segment is lessthan a first threshold; wherein the second iris recognition is performedbased at least in part on the determination that the first confidencescore associated with the first segment is less than to the firstthreshold, the second iris recognition comprising: identifying a secondplurality of cardinal points around the eye based on a second eyemodeling of the detected body region; performing a second irissegmentation to identify a second segment of the iris within the bodyregion based on the second plurality of cardinal points; and determiningthat a second confidence score associated with the second segment isless than a second threshold, wherein the third iris recognition isperformed based at least in part on the determination that the secondconfidence score associated with the second segment is less than thesecond threshold.
 17. The method of claim 16, further comprising, inresponse to the first confidence score associated with the first segmentbeing above the first threshold, identifying a subject of the imagebased on a result of the first segment.
 18. The method of claim 16,further comprising, in response to the second confidence scoreassociated with the second segment being above the second threshold,identifying a subject of the image based on a result of the secondsegment.
 19. The method of claim 16, further comprising: identifying athird plurality of cardinal points around the eye based on a third eyemodeling of the detected body region; performing a third irissegmentation to identify a third segment of the iris within the bodyregion based on the third plurality of cardinal points; determining thata third confidence score associated with the third segment is less thana third threshold; and determine that the third iris segmentation failedbased at least in part on the determination that the third confidencescore associated with the third segment is less than the thirdthreshold.
 20. The method of claim 16, wherein criteria used to definethe first threshold and the second threshold are the same.
 21. Themethod of claim 15, wherein the first iris identification is applied atmore closely spaced image locations than the second iris identification.22. The method of claim 15, wherein detecting the body region in theimage comprises performing at least one of a face detection or an eyedetection.
 23. The method of claim 19, wherein the first segment, thesecond segment, and the third segment are each defined by an innerperimeter and an outer perimeter.
 24. The method of claim 19, whereinthe first confidence score, the second confidence score, and the thirdconfidence score are based on at least one of the first segment, thesecond segment, or the third segment, respectively, or an iris codeextracted from the first segment, the second segment, or the thirdsegment, respectively.
 25. The method of claim 19, wherein the first eyemodelling, the second eye modelling, and third eye modelling areperformed the same.
 26. The method of claim 19, comprising, in responseto the third confidence score associated with the third segment beingabove the third threshold, identifying a subject of the image based on aresult of the third segment.
 27. The method of claim 19, furthercomprising, in response to the third confidence score associated withthe third segment being less than the third threshold: selecting ahighest confidence score from among the first confidence score, thesecond confidence score, and the third confidence score; and selecting asegment from among the first segment, the second segment, and the thirdsegment that produced the highest confidence score from which an iriscode is extracted.
 28. The method of claim 19, further comprising, inresponse to the third confidence score associated with the third segmentbeing less than the third threshold, indicating that the iris is notdetectable within the image.
 29. A non-transitory computer readablemedium storing instructions to, when executed by a processing unit:detect, with an image sensor, a body region comprising an iris of an eyein an image; performing a first iris recognition of the iris; inresponse to the first iris recognition failing, performing a second irisrecognition, the second iris recognition comprising a first irisidentification on the image at a first detection scale; and in responseto the second iris recognition failing, performing a third irisrecognition, the third iris recognition comprising a second irisidentification on the image at a second detection scale, wherein thefirst detection scale is relatively larger than the second detectionscale.
 30. The non-transitory computer readable medium of claim 29,further comprising instructions to, when executed by a processing unit:in response to the third iris recognition failing: determine which ofthe first iris recognition, the second iris recognition, or the thirdiris recognition produced a highest confidence level; and extract aniris code for the image using the one of the first iris recognition, thesecond iris recognition, or the third iris recognition with the highestconfidence level.
 31. The non-transitory computer readable medium ofclaim 30, further comprising instructions to, when executed by theprocessing unit: in response to the third iris recognition failing,return an indication that the iris is not detectable within the image.32. An image processing system for iris recognition comprising: an imagesensor to acquire an image; and a software processing unit to executeinstructions stored on a non-transitory computer readable medium to,when executed: obtain, using the image sensor, the image; detect a bodyregion comprising an iris of an eye in the image; perform a first irisrecognition of the iris; perform a second iris recognition, the secondiris recognition comprising a first iris identification on the image ata first detection scale; and perform a third iris recognition, the thirdiris recognition comprising a second iris identification on the image ata second detection scale, wherein the first detection scale is differentthan the second detection scale.
 33. The image processing system ofclaim 32, wherein: wherein the first iris recognition comprises:identifying a first plurality of cardinal points around the eye based ona first eye modeling of the detected body region; performing a firstiris segmentation to identify a first segment of the iris within thebody region based on the first plurality of cardinal points; anddetermining that a first confidence score associated with the firstsegment is less than a first threshold; wherein the second irisrecognition is performed based at least in part on the determinationthat the first confidence score associated with the first segment isless than the first threshold, the second iris recognition comprising:identifying a second plurality of cardinal points around the eye basedon a second eye modeling of the detected body region; identify a secondsegment of the iris within the body region based on the second pluralityof cardinal points; and determining that a second confidence scoreassociated with the second segment is less than a second threshold;wherein the third iris recognition is performed based at least in parton the determination that the second confidence score associated withthe second segment is less than the second threshold, the third irisrecognition comprising: identify a third plurality of cardinal pointsaround the eye based on a third eye modeling of the detected bodyregion; perform a third iris segmentation to identify a third segment ofthe iris within the body region based on the third plurality of cardinalpoints; determine that a third confidence score associated with thethird segment is less than a third threshold; in response to the thirdconfidence score associated with the third segment being less than thethird threshold indicating that the iris is not detectable within theimage.
 34. The image processing system of claim 32, wherein: the firstiris identification is performed on less than all of the image of thebody region; and the second iris identification is performed based onsubstantially an entirety of the image of the body region.